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Accountability, trlith in testiSi; 6ompetence-based' curricuia--fhese recont 
terms in the educational lexicon illustrate today's concern for evaluatlPB 
performance in education organizations. But whose perforaance: student? 
teacher? administrator? textbook designer? test c.nstructer? Recent debates 
ever the nature of testing have not clarified i - answer to that question 
primariiy because the perfonaahce of ^ actor may cT interest to someone 
at some point. Tbia pape^'r attempts to address the question at ^5 middle range 
level; its purpose is to set standardized educational testing within a cOntflXt 
of adminiatfati¥e performance infbiraation. Tbua it ignores such issues as the 
technical valfHity of tests, the psychological repercussions of test trauma, 
Md the social inplications of tracking based on tests. Instead, it argues 
that standardized educational tests produce only one fom of iifomation 
present and possibly used in school organization administrative performance 
information systems.: It- begins to specify the nature of such a system, 
assesses the role of test information within it-, and speculates on possible 
functions and dysfunctions of such a system. 

; AMIHISTSATI7E PEfiFeBSAHCE INFORMATION SfSTO! 

Administrative performance information is infomation collected by and for 
the administrative structure^ of organization about the performance of 
various coi^nents of the organization. In the case of educatioh 
organizations, this includes information collected by and for administrators 
^^^""^"^ °^ students,- teachers, and administrators. At tines 
this information may ii^i problematic situations irithin the organization «nd 
lead .to changes in behavio^. - for example, it could lead to changes in 

.personnel aiii^ent and compensation, instructional activities, or 
crganizational structure. . At 5ther times, it may simply function as a 
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monitoring device ^ to assure recipients that perforiiiance is generally 
adequate; 

To the extent that coliectidni analysis^ and use activities are 
predictable, repetitive, and related, they constitute an administrative 
performance information system. Three characteristics of such a system are 
notable: characteristics of the information collected; the intended users of 
the collected information; and the actual uses to which the information is 
put. Let us briefly review each of these three characteristics. 

Information collected in an administrative perfdrtPance information system 
varieo on several dimensions. Indeed because the typical system includes 
multiple forms of information^ many dimehsibhal values are present in any 
system, even though some may predominate over others. Information comes from 
various sources . In the case of educational organizations, students, 
teachers, principals, and perhaps parents, coinprise ma^df sdurces. It is 
generated through ^various coliectibn modes . Siese include personal 
observation, reports df dbservations made by others, and systematic, 
"objective" measuring instruments such as attendance records and standardized 
forms or tests. The timing of collection can vaiy, from frequent to 
infrequent, regular td irregular. The information concerns various subjects 
of performance. Siese may range from expenditures to attendance to learning 
td ene**^ conservation. And the bandwidth of the information channels can^' 
ran^e from«' very broad to very narrow. For example, it can range frdm a 
personal report of classroom observation complete with descriptions of the 
physical space, the kinds df instinictidnal activities^ and the apparent 
engagement of students, to a daily attendance report. The former represents a 
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much wider information channel thail does the iatter. 



If the characteristics of the <^ information can vary^ so tod car. it<? users. 
An administrative performance information system can provide irfortujjticn for 
any member, client, or constituent of the organization. It is imjor^art >o 
note that administrators themselves should not be considered the only 
potential users of information generated by this system. Their involvement 
may extend only to sponsoring the collection of information at the request of 
or to be xised by others within or interested in the organization. 

Probably the most commonly employed metaphor fcr the use of performance 
information in organizations comes from cybernetic theory. . The- metaphor 
. suggests that when information indicates the organization is malperforming, 
^ corrective action is taken to bring organizational performance back into line. 
Thus the preeminent asst:med use for performance information is to improve 
perfonaance. This metaphor underlies common rationales for management " 
information systems and management by objectives (Ackoff^ 1967). It also 
underlies much of the current cbncera about the use of evaluation information 
in social programs (Suchman, ^9€^)* 



Casual application of a ^yb^riietic metaphor to drganizatidhs ignores three 
important features of any tme cybernetic system, however, and therefore leads 
to faulty assumpitibhs about organization behavior implied by the metaphor; 
The first feature of a ^true cybernetic system is that if information reveals 
performance inadequacy, corrective strategies are atitomatically implied within 
that informations In the classic case^ if a thermostat reading is too low or 
too high, the solution to that problem— turning a heater oh or off--is 
automatically implied and carried but. This feature of cyberaetic systems can 
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lead organizatibnai analysts to assumi that performance infonnatibri in 
organizations always implies or carries with it corrective strategies. Tbe 
second ^feature of a true cybernetic system is that it is a closed system, one 
in which the producer, collecter, and consumer of infbnnation are 
self-contained within a feedback loop. Siis feature, can ii-aU organizational 
analysts to assume a necessary similar connectedness within o^anizatttins, 
assuming that the producers and collectors of performance information should 
also be. its consumers. The third feature of a true cyberaetic system io that 
information use need not be equated with corrective action ; information :?an 
also be used to indicate adequate performance. Again in the case of the 
thermostat, information indicating adequate temperatures is net ignored 5 it is 
registered as a sign of adequate performance. Too ofteii analysts of 
organizations equate' "use" with corrective action, assuming that if the latter 
,^ia not occiirring, neithar is the former. 

limitations of a cybernetic metaphor in understanding the use of 
performance information in organizations highli^t three propositions about 
any administrative performance information system. 

1. Inf ormation_ use should hot be equated only with *borrective action. 

2. Information users atoul^. "not be equated only with information 
producers or collectors. 

3* Information nay or inay not car,ry with it implicit strategies for 
corrective action. 

Given these, propositions we can detail three general uses of information in an 
adainistrative performance inronnation sjstem, building on the work of Simon 
(1971, 1973). - ■ - ; 

Information in an administrative performance information system can be used 



ERIC 



8 



ERIC 



to serve a scbrecard ftanctiori. (Note that r consonant with proposition one, 
this does not imply aiay corrective action.) It serves to let users knbv that 
things are more or less on course, often bj incorporating implicit or explicit 
^conparatiye standards. These standards may be nistorical, lending themselves 
to the construction of more or less systematic moving averages. Or they may' 
be normative, lending themselves to cross sectional comparisons. Often this 
information is quantitative and, at the administrative level, appears in- 
aggregate form* For example in police departments, monthly arrest or 
complaint investigation records • are a form of aggregate performance 
information that serve a scorecard function, letting the commander know that 
things, are pretty much on target. Score carS information, whose collection is 
sponsored by administrators, can also be used by persons outside the formal 
idministrative structure. If this information is disseminated to the outside 
^ world it may be labeled as -TPR," "accountability inrormation," or 
-federally-mandated evaluation results.** In any case, it can serve to let its 
recipients know that things are more or less on target. The parent receiving 
a child's report card, the federal gbverament receiving an evaluation report, 
or the stockholder receiving ah annual report are all most likely to use that 
infarmation in a scorecard mamer^ (Note that we are saying^ consonant with 
proposition two, that the producers and collectors of information need not be 
its only users.) 

* ■ ' 

Occasionally, however, information in the administrative performance 

Information system signals a problematic sj^tuation. - That is, the second way 

. r 

in i&ich administrative performance information system information can be used 

is to direct attention, in effect it signals that tolerance levels have been 

breached and solution strategies should be sought. (Sere, consonant with 
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proposition 'three, ve note that attention dirictdng infSrmatioii need not imply' 
a specific sblutidn strategyO Qncfe again, infom^tibn may bT^used to direct 
attention by either those inside or those outside the actual administrative 
structure of the organization. If by the. former, the result is often staff 
.meetings, first to verify that a problem does in fact exist' and second to 
explore vays to solve it (Kiitzberg, 1975)- ir by the latter, the result is 
often further attention directing efforts designed to convince those- within 
the organisation that a problem does exist. 

The third way in which administrative performance icforaatibn system 
inTormation can be used is actueiiy to solve performance pro'hlems. Given the 
^ la^e volume of infoimstion flowing through an administrative performance 
information system, this probably turns out to be the least prevalent of. the 
three classes of use. In order for information to be used in solving a 
performance problem, it must be relevant and appropriate (characteristics of 
the informationX and in the hsnd& of the problem solver (characteristics of 
the user). , " ' 

This, section has outlined so>ie of the major . general features of any 
administrative performance inforaation system. In order to better specify 
their characteristics and functions, it is necessary to investigate 
empirically how various classes of information and users interact within real 
organizations. 

Standardized tests are one component in the administrative performance 
informatiotfc system of any educational organization— public or parochial 
district or private school. Through an exploratory field study we have begun 
to investigate the role of standardized tests and their general contribution 
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to the overall iaaffiinistrative per^rmance inforaation system in eaucational 
brganizationi. The following two sections of the paper aescrit5e^the field 
study and the contribution of s'tandardized tests to the ichObl organization 

e 

administrative perfcilnance inrormation system. 

FIELE STUDY DESIGN - " * 

The purpose of this study was to provide systematic descriptidna of central 

S) . 

office ..perspectives on standardized testing. Public (urban and suburban), 
parochial, and private school systems were investigated. Because this was an 
exploratory study, we did not sample randomly from the entire population of 
school organizations; rather^ we simply solicited volunteer school 
organizations from all those operating within one large county in . ¥estem 
Pennsylvania. ^ta were collected during interviews with central office 
administrators having involvement with standardized testing in one urban 
district, one parochial system^ eleven suburban districts, and five private 
schools. (Table 1 displays characteristics of the organizations in our study 
and compares them with the larger population.) In every case, the pefH^;g|^in 
charge of testing within the organization was interviewed. Often, other 
central office personnel who have some contact with test scores were 
interviewed as well. J^^---^' 

Data were collected by extensive, personal interviews. Sie interview 
schedule covered many aspects of testing programs such as purposes, 
organization, uses, and users. The questions were open-ended so that 
respondents could answer in their own words and not be constrained by 
predetermined categories. These answers comprise the bulk of our database. 
_ _ _ _ _ • - - 

Data analysis was^ undertaken wi-fc, the prr.jaiy goal of distilling as 
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accurate and full a picture as possible of testing as seen by central office 
adminiatratdrs. We tallied every response to each question knd deveibped 
groupings based on the responses; t^ing to preserve the spirit of the 
individual res^nses (Slaser and Strauss, 1967). The groupings were then 

. interpreted in teraa of their frequency and their eihaustiveness. Four levels 
of analysis were used: the school system, the organization (ichooi or 
district), the respondent, and the i-ecponse. Most questions were analyzed 
only at the level most relevant to our objectives, and many questions only 
made sense at one particular level. For example, -the presence or absence of a 
testing office vas noted only at the organizational level; purposes were 
counted l^^the response. From these data ire have attempted to describe 
standardized testing as our ^respondents see it, and then to compare and 

» analyze the descriptions to better understand the rrole of testing in the 

tfehtral offices of school organizations. 

STAHDAEDIZED ^STISG IN'Tm ABiranSTEATITE 
PKRPOEHAHCE IHKlEftWieN STSfffl 
Scope an d^ignificance of Testing Programs 

Every school organization in each school system supports a standardized 

testing program. Table^ 2 presents data on selected characteristics of testing 

programs in each system. Overall ^ the average niunber of standardized tests 

administered per school' organization is five. The public--urban iind 

^ . . - ■ 

suburban—systems employ more tests on average than do the private or 
parochial systems. However, the range in the number of tests used by suburban 
and private „ schools are identical; some suburban schools only use t^o 
standardized tests and some private schools use as Mig* as nine. 

The standardized tests used by scKobls fall into three aain categories: 
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achievement tests, ability tests ^ and vocational aptitude tests i (Ve ^impiy no 
ratification of a distinction between "ability" and 'achievement" tests; this 
-• distinction- comes from school administrators themselves;) Overall, 
achievement and ability tests were reported with almost equal frequency 
followed by vocational aptitude tests. Ail school organizations emplby at 
least one achievement teat: moreover^ achievement tests ^.outnumber any other 
kind of tests ±n the parochial and private systems. -The urban district 
employs an equal number of achievement, vocational aptitude, and other tests 
while ability tests predominate in the suburban system. Interestingly, the 
puKLic systems : report using more different kinda of tests than the parochial 
' or private systems and are the only systems ^ to report using vocational 
aptitude tests. ' . ^ 

School organizations administer standardized achievement tests daring the 
fall, spring, both, or at mid-year. Spring is the most common time for 
achievement testing. Most private and suburban school organizations test in 
the spring. The urban district tests during both the spring , and fall- as does 
one suburban district. The parcchial system and one private schooi test at 
aid-year. v 

When school organizations administer achievement tests, they may test every 
student or some subset, for example, students in every other grade. Most 
school organizations use achierement tests in the grades up to high school, 
with some teeting through high school as >^^ell. The most prevalent testing 
pattern is to te.st every sttident starting in grade one or two through grade 
eight. Only one school oi^anization gives achievement tests in every other 
gr^de* 

o 
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The process by which a school organization selects a new test may reveal 
some general attitudes about the role of testing within the organization. 
Often a formal committee ic established for this purpose; less freqii'ehtly a 
group is called together to ratify the recommendations of someone who has 

researched a number of different tests. Overall^ Just"* slightly more 

- ^_ 

organizations exclude teachers from their selection committees than include 

_them. In none of the school organizitibns were test selection decisions made 

by a single administrator. In the urban and parochial systems, committees 
_______ I 

composed of central office administrators and pupil services personnel select 

new tests* In about half of the suburban and private school organizations > 

these cdmmittees al^o include faculty; in the school organizations that do 

not; test selection committees are similar to the urban and parochial 

committees. 

Three categories were found to account for most of the criteria used in 
test selection: test content and design, information produced by the test and 
it^ potential use, and monetary and time costs. Test content and design, 
which includes curricular validity, was the most frequently mentioned category 
of test selection- criteria in each system. This was the only category of 
criteria reported by the parochial respondents. In the other three systems^ 
the next, most frequently cited criterion was infbmatibn produced by the test 
and itW potential use within the brgaifeatibn. For example, this category 
includes statements such as "the kinds of reports the publisher provides.** 
The least frequently mehtiohed ciriterion was monetaiy 'and time cbsti For 
instance, '"the test should be easy to Administer" and "the length of the test" 
are statements about the time costs of' tests i (These last two categories were 
mentioned ^ith equal frequency in the urban system j In general, test content 
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and design seems to be the 'most imi>ortant, and prevalent category of test 
selection criteria; 

- Indicators of the orgahizatiohaT status of testing programs trithin school 
systems are shown in Table 3- Most school brgahizatiohs do not have a formal 
testing office. Only in the urban district and one private school were 
testing offices found. In school brgahizatiohs irLthout a testing office, 
responsibility for testing is usually assigned to a central office 
adminis^ ratpri -(An exception to this was found in one private school where 
.testing was the responsibility of each teacher.) The school systeis differ in 
terms of where they place responsibility for testing, but the level at which 
responsibility for testing is placed within the organization is fairly 
similar. In suburban school districts, testing vas most often found under the 
direction of a pupil personnel administrator, and the next most frequent 
arrahgemeht was the sharing of responsibility between a pupil persomel and 
another central office administrator. General administrators are given charge 
of testing less often than any other central office administrator in suburban 
school districts. Regardless of the functional area, the person in charge of 
testing, is likely tO: be ah assistant superintendent (or equivalent) and report- 
directly to the superintendjait.- - In the parochial system, the elementary 
testing program is the responsibility of an instructional administrator who 
reports to a general admihistratbr^ and at the secohdaiT: level a general 
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admihistratbr is in charge who reports to the superintend eht. In four of the 
five private schools, testing is the responsibility of a general administrator 
such as the Bire5tor of the Middle School who then repbrts to another general 
administrator^ the person in change of the entire organization. In the urban 
system, the director of testing is a staff position and reports to an 
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instSctional administrator. fhe urbai district is the dniy i5h6ol 
organization where this pattern of assigment of responsibility was found. In 
each system testing is located in a different functional area; however, the 
person in charge of testing usually reports to the person in charge of the 
entire organization. There is limited evidence to suggest that 'responsibility 
for testing resides at a "lower" level in the organizational hierarchy when 
there -xisti ? formal testing office than when there does not. 

The number of central office personnel who participate in testiii programs 
klso Vari.es. The mOdal number of staff involved with testing for all s^ terns 
is two. The public and parochial secondary school organizations have 
relatively fewer personnel involved compared to private and parochial 
elementary organizations. 

^ 

When central office administrators are asked to justify the existence of 
their testing programs, they do so in similar ways across the different school 
systems. Fote that these justifications Say or may not have any cohnectibn 
irith how test scores are actually used; these are normative statements. 

Overall, the three most frequently reported justifications for testing were 
individual student dia^qsia-^^and placement, program evaluation, and 
achievement measurement. (See Table 4.) Tiro purposes were common to all 
systems: individual student diagnosis' and placement and reporting to outside 
audiences. In the; urban, parochial, and private school systems, 
Bttjc'ent-oriehted pur^ses outnumber program-oriented pui^oses. In the 
suburban system, these two categories of purposes are reported with equal 
frequency. Program monitoring was unique to the suburban system as was school 
adiission to the private. Of p?og?ai-oriented purposes, internal prograi 
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evaluation and program evaluation for outsiders were most cOmmOn. Apparehtiy, 
central office administrators View testing as serving student-oriented 
purposes primarily, and program-oriented purposes to a lesser extent. 

The relative salience of standardized testing to central : office 
administrators was estimated from several indicators.^ (See Table 5) As noted 
earlier, few school X)rganizati6ns Save formal tes'^ting offices; furthermore, 
only three respondents ^hought their organizations could use a testing office. 
Excluding the urban system, testing occupies less than V0% of the time of 
whoever is. in chaise. It must compete for attention witth a plethora of other 
responsibilities ranging from transportation to budget development. Also, 
testing expenditures are relatively small compared to other organizational 
expenditures, such as transportation oF'maStenance. - In no district did they 
exceed five dollars per pupil. Furthermore, most respbhdehts felt that their 
testing programs were ' adequately funded. Taking all these indicators into 

r 

■ account, it seems plausible to assert that testing is not very salient to 
central office administrators^. 

Actual Use of Test Scores from^ a_eentrai Office Perspective' 

Reports about who finds test information to be most useful- also indicate 
that central office jadmiiistrators believe most of the benefits of testing 
accrue to others, primarily building level actors. Table 6 shows that in all 
systems central office administrators named building level actors as those 
finding test information m^t useful. Only in the private system is the 
number of responses citing central office administrators close to that of 
building level actors. Among the building level actors, teachers are believed 
to find test information most useful; of central office administrators ^ it is 
those irith instructional responsibilities! nOt those in pupil personnel as 
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might have been expected. Central office administrators view those actors who 
carry out student-oriented purposes (teachers, counselors, and principals) as 
the group who find test information most useful. . . . 

Another indicator of) who central officis administrators perceive as finding 
test information most salient is their report as to whom would miss testing 
the most if it were abolished. Overall, central office personnel believe 
building level personnel would miss testing the most ±i it were abolished. Of 
any single group of actors, teachers were thought to be those who would miss 
it most followed by students, parentis, and the community. In the urban 
system, central office persohhel as a group were mentioned more frequently 
than building level peraonnel, and general administrators and teachers were 
the most frequently reported actors. In the suburban system, building level 
personnel were reported most often as those who would miss testing the most; 
specifically, teachers and counselors account for almost half of the suburban 
responses. In the parochial system, instructional administrators and 
counselors were each mentioned once. In private schools, parents* students* 
and the communi^ were most frequently cited as the actors who would miss 
tasting the most if it were abolished. * Teachers and general administrators 
were also mentioned by the privsfe system respondents. These findings suppKDrt 
the nbtibh that central office administrators do not perceive themselves to be 
primary users of test infbrmatidn. Prom their vantage* the intensive use of 
test information occurs at the building level. 

Central office administrators justifications for having a testing program.- 
and their beliefs about primary users of test information d«nonstrate that 
central office administrators- do not view standardized test information as 
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very salient $b their probleias. Rather^ they feel it is necessary and 
iippbrtant to provide test information to others in the organization who tend 
--to use test information at the individual student lev^l. However, central 
office administrators do^ not ignore test information altogether. 

Central office administrators were asked to report how they interpret tWt- 
information, that is, the ''rules of thumb'* they use when reviewing a new set 
of test scores. ^e responses collected suggested three dimensions along 
which interpretation rules can be classifieds "Ttie first is inter-time versus 
Intra-time period measures; that is some respondents make explicit comparisons 
with scores from previous years, iSile others do not. ^e second is 
comparative versus non-comparative measures. 1 The third is, for those who do 
make comparisons, internal versus external reference groups. ^ por ail 
systems, intra-time period measures are the most common and within this 
category non-comparative measures were the most; frequently reported. (See 
Table 7.) Looking at the systems individuaiiy, it is interesting to note the 
most frequently reported interpretation measures for each systems In the 
urT>an district, ^comparison with an external reference group (national norms) 
Is the moiit cbmmbn interpretation rule. For the suburban and private systems, 
non- comparative measures predoniinate and for the parochial system only 
cbmparisbh with; an internal reference group was reported. 

'^Son-comparative measures are reflected in statements such as "I' look at 
building averages" and "I look at classroom summartes." Administrators 
probably have an implicit reference group in mind iffien using these measures, 
however, they are nbt revealed in their reports. 

2 — . _ __ _ _ 

Internal reference groups are groups of students within the 3ch66i 

organization such as grades* buildings, or subjects, fhe most common exteraal 
reference group measures are natibnal norms. 
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Occasionally test scores do stimulate action on the part of central office 
administrators. Respondents were asked to report irhat kinds of test scores 
ifbuld cause or, have caused central office action. Overall^ comparisons wifch 
• external reference groups were cited most often as the category of measures 
that would lead to central office action and vas the only category common to 
^ all systems in contrast to the data on test^ score interpretation* The most 
commdn use of test information by central office administrators is for 
curricuiar evaluation. in one district^ scores on map reading , in geography 
were low. Investigation suggested this was due to lack of empliasis on 
geographic terminolb^; so terminology was subsequently empbaaized. In 
another district, all elementary students scored low on listening skills. The 
Assistant • Superintendent erf Instruction asked the principals to discuss the 
problem with their teachers and to propose solutions which resulted in some 
. curricuiar changes. In a third district, fourth grade reading scores 
— consistently lower than natioiial norms caused administratora to devote more 
resources to fourth grade reading materials and in-service training, ^es^ 
examples are tjrpical of the way test information is used by central office 
.administrators with regard to educational programs. Test information does not 
usually define or specify a ^o^\^ completely, but instead indicates the 
general area where a problem exists. Rarely is change made on the basis of 
test information alone; cdrrobdratibn frbo some other source is usually 
sought. For do test scores automatically imply solutions; these must be found 
elsewhere. Often alterriative sources hold information that is richer and more 
iiBaediate for the central* office administrator. Is a result, the only 
exception we found tb this condition occurred in the urban district where 
adninlstrators decided that perhaps one reason scores were low was because 
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stuaents dia not know how to take tests; As a result, cominefcially pfeparea 
lesson plans on test taking irere purchased for eveiy elementary classroom in 
the district i : ■ . . 

In summaiy^ central office aaministrators believe that test infomation is 
most useful to buiiaing level actors for student-oriented purposes. Teachers^ 
principals^ 'and counselors are expected or assumed to find t^st ihformatidh 
useful for individual student diagnosis and program placement and achievement 
measurement. For' central office activities^ test information seems to be 
perceived to be most relevant for curricular evaluation. Purthermbre^ at the 
central office level, test information ' is primariiy used for scorecard 
purposes, such as for program monitoring and reporting to outside audiences. 
Test information is rarely a sole basis for miartg decisions. Often other 

sources and types of information seem to be more salient for central office 

------ 1 ^ 

problems and decision making. 

Perhaps central office adminisiratbrs* preference for information of a 
different nature than test information is best illustrated by looking at how 
these educational organizations evaluate their overall perfdrmahce. Personal 
observation^ teacher observation, parental and community feedback, and the 
achievements of students and' graduates were, all felt to be more indicative of 
overall organizational performance than were test scores. Yet, although test 
scores were reported as playing a minor ^ if any, role in overall 
organizational perforaance, they were the only type of information common to 
all of the organizations studied. 5£e prevalence of 'teat info™ationi||^-. 
descriptions of the administrative perfdraance information system may be due 

to the availability of test information in all educational organizations: and 

J — 
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environmental influences such as external reporting requirements, rather than 
to their perceived utility to central office administrators • 

-■ - - ~— DlSCUSSi^^ 

The :relatively low use of test information j)y central office administrators 
suggests that ttst information may not possess desirable qualities from the 
perspective of the central office. Two dimensions along which performance 
infbnaatibh can differ are immediacy and attachment. Immediacy is the state 
of being useful right away. Defining problems^ proposing aolutions, and 
timing are all avenuei to immediacy. Attachment is association with a 
specific^ identifiable person or group. Presentation and content contribute 
to the degree of attachment of information. Both of these concepts should be 
thought of as continua. What, if any, trade-off occurs between them is not 
known; however^ it would seem reasonable that information relatively high on 
both dimensions would be preferred to that rated aow dh both. Host test 
information may be viewed by "central office administrators as relatively low 
on both scales. 

In contrast, personal observatibn^ personal reports by facility, staffs and 
students, and parental and community feedback would seem to be relatively high 
on immediacy and attachment*^ ^Personal contacts^ especially complaints ^ often 
specify a problem and what should be done about it. Either' of these alone 

caiTTies more information that is immediately useful and salient to the central 

— « _____ >. 
office administrator. - s. 

Item analysis, particularly when combined with test (J^sign/ may alleviate 
some of the lack of immediacy of test information. Publishers are. now able to . 
supply clients with reports as to the level of mastery of specific skills; and 



concepts. In addition^ they provide curriculum guides that contain ready 
remedies when test scor^ indicate low comprehensibn or masteiy. In a senses 
the test scores cross-index problems and solutions. This "system" niay 
increase use of test ihfbnnatibn by central office administrators concerned 
ifith instructional strategies by increasing the immediacy of -the information. 
This Iras not investigated during this project and is speculation; however, the 
two largest organizations have purchased of are cbntemplating the purchase of 
this tjrpe bf testing program and service during the course of this project. 

In sumxnaiy, central office administrators do not view themselves as major 
users of test information. They believe test informatibn ds most useful to 
building level actors for iSbm they purchase the information. Occasionally 
central bffice administrators do use test infomation themselves^ it is 
"nisualiy in connection with curricular evaluation and usually not, the bnly 
information invblved in the decision. Sather, a variety of other sources of 
information are drawa upon and this bther infbrmatibh receives greater 
consideration particularly with regaxd to overall organizational performance 
assessment. This preference may .be due tb attributes bf the information: the 
amount of immediacy and attachment they possess « This, hbwever^ awaits 
empirical investigation. 
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Table 1 i 



^ Repr es en tatlvetiess. of Sc hdi?! Orj^anlzatlons In 
"lield Stu^^Cejparlson vith County^ State and Katlonal Bata* 



Average 
Mbiii^t 


Urban 

48,795 


Suburban 
4962 


Parochial 
19,012 


Private 
314 


^. 

Public Schools 
in Allegheny 
County 

5528 


'Public Schools 

_ in 

4306 


Public Schooli 
in the U.S. 

2738 


5 Son-Public 
1 Schools in 

276 


-XHinority 


49,08 


4.4 


I 

\ 

V 


/ 

) 


* 

N.A, ■' 
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TQtal 
Budget 


$143,525,000 


$11,477,258 




- 


$10,112,565 


$7,207,295 


$4,629,250 


$518,880 


Average 
Per Tu^il 
Expenditure 


$2941 


$2309 ■ 




m 


$1833 


$1852 


$1638 

1 


$1880 


Niiber o! 
Schools in 
Organization 


101 


• 

a' 

i 


57 


1 

1 






: — t 

1 





*Data for study participants are for the, 1978-1979 school year.' County, State M National data 
are for the 1976^1977 school year. ; 
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Table 2 



Characteristics of Testing Programs 

Testing Program ' V : 

Characterirstics — - School System Type 

Urban Suburban^ Parochial Private Total 

.Number of Standardized; - ; 

.tests * ^ ' ^ 

Average Number of . ^ 

_ Tests il 6 3 n _ ; 5 

^ Range . ^ - 2-3 - 2-9 2-11 

General ebntent of ; - 

tests 

Achievement 3 20 ^35%) 3 11(52%) 37 

AbUity _ ' 2 25(43%) 8(33%) 35 " 

Vocational * 3 12(21%) - ^ 15 

Other ^3 1(2%) - 2(10%) 6 

> - . - 

Tests n - 11 58 * 3 21 93 

Time at Which 

Achievement Tests 
— Are Administered ^ ^ . - 



Fall and Spring 1(100%) 1(9%) ^ - « 2(11%) 

^^11 - 3(27%) ' - 1(20%) 4(22%) 

Spring ^ ^ 7(64%i^ 3(60%) 10<56%) 

Hid-term ^ - . 1(100%) 1(20%) 2(11%) 



Organizations n- 1 11 1 - 5 18 

Grade Levels at Which • ' • 

Achievement Tests * - • " 

Are Administered • : - 

Ctae_or fab - Eight ^ 1(100%) 9 , 1(100%) 3 14 

Every Other Grade ^ 1 \^ ^ . * 1 

• # . 2N/A 



Organizations n - 1 15 



Data are for ten of eleven districts^ 
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Table 2 cont. 



Test Selection Process. 



Participants " 

: Single Administration 
Go^ittee of ^ 

: Committee of 

Administrators and 
Faculty 



Organizations n 
Oriteria 



Urban 



School 



Suburban Parbchia 



1(100%) 6(55%) ,1(100%) 
- ■ 5(45%5 



11 



Test Content and 

Design 2(50Z) 
Infdrinatlbn Provided 
and its Potential 

IJse _ _ 1(25%) 19(28%) 

Costs (Mbnetary and 

T^e) 1(25%) 16(24%) 

Other - 2(3%) 



31(46%) 2(100%) 



js n * 



68 



* Usually includes pupil ser^rices personnel. 
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Table 3 



Drgahizationai 



liHHSlndlcatbrs 



• For maJ^Testinp Qfft,-^ 

^^oslti on Responsible 
General Adifilnlstratbr 
Itiatructldnal 
_ Administrator 
Pupil Personnel 
Admitiistra'tor 
Pupil Personnel 

tflth tnstri Adin. 
on Gen, Adm. 
Other 

Director of Testing 
Repbr^B t-». ^ 

Superintendent 
^•ral Administrators- 
instructional ^ 

Administrator 
^Pil Personnel 

Administrator 
Other 

Number of Personnel 
p^ ojyed with Testing 

2 
3 
4 

5 5 up 



^ehbbl System Type 



Urban Suburban Parochial Private 
l(l6o%)'o(0%5 0(02) 1(20%)" 

secondary 



1(100%) 



1(11%) 
2(22%) 
5(56%) 

3(33%) 



1(100%) 



9(82%) 
1(9%) 



1(9%) 



1(100%) 1(11%) 
5(56%) 
' 3(33%) 
1(11%) 
1(11%) 



drganizitio 



»as n 



11 



(50%) 
el erne 
(50%) 



i 



4(80%) 



1(20%) 



|gg|dary 

fg^^ntary ^^^^^ 



llgg^dary 

1(20%) 
- 1 (20%) 



1* 



'iotal 
2(11%) 

5.5(3iZ) 
2.5(14%) 
5(28%) 



3(17%) 
2(11%) 



9.5(53%) 
6.5(36%) 

1(6%) 

1(6%) 



2.5(14%) 

6(33%) 

4(22%) 

2(11%) 

3.5(19%) 



18 



f. J^*^°^^^^ ^7s^^ is considered "one organization. Each educational division 
vexementary or secondary) is counted as one-half. 
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Table 4 



Purpos e s of or Justif Ica^idris fbf Standardise d 
Testing a s Rep orted. by Central Office Administrators 
(Multiple responses possible) 



• Urban Suburban Parochial Private Total 

Student Oriented 

Admissions 2(25%) 3(3%) 

Diagnosis & Placement 7(29%) 18(29%) 2(40%) 3(25%) 30(29%) 

Abiiigy Measurement 6(25%) 1(26%) 2(17%) 9(9%) 

Achievement ^ 

Measurement 7(29%) 10(16%) i(8%) 18(17%) 

Provide Scores to 
Principals or 

Teachers 3(5%) 1(20%) 4(4%) 



Program Oriented 
Instructional 

Program Monitoring 4(6%) 4(4%) 

Instructional Program 

_ Evaluation 13(29%) 2(17%) 20(19%) 

Reporting to Outside ; 

Audiences 1(4%) 9(14%) 1(20%)' 1(8%) 12(12%) 

Teacher Evaluation 3(13%) 1(2%) 4(4%) 



responses n - 24 63 5 I3 
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Table 5 



Indicato t^ of Relative Salience 
^ of Stand ardized Testing to Central 

Office Administrators 

S a lience Indicator s __ . School System Type 

Urban Suburban Parochial Private Total Sainpl 

Presence of Testing 

__ Office? (By Organization^ (106%) 0(6%^ 0(0%)' 1(20%) 2(11%) 
Should there be one? 

(By respondent) - 2(6%) 0(0%) 1(77%) 3(5%) 

Amount of Time spent on . " 
Testing by "Persdit in 
Charge of Testing''^:. 

(By respondent) 100% 10% 10% 5% 

Approximate Per Pupil 
Testing Expenditure 

(By district) $3.75 $1.96^ $^.25 $2.80 - 

Is Testing Adequately 
Funded? (By 

Respondent) 16(84%) 28(90%) 1(50%) 5(83%) 50(86%) 

Organization n« 1 ii 1 5 

Respondent n » 19 31 2 6 58 



^Figure represents the amount of time spent by the modal "director of testing." 

^Figure reported is_ the average based on 
districts were unable to supply figures. 



^Figure reported is_ the average based on data for six Sis trie ts. Other isuburban 
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Table 6 

_____ ' ' 
Salience of Test -^trf <)rmat j.c>n to Various 

Actors as Perceived by CPAs 

(Multiple responses possible) 

Salience Indicator School System type ^_ 

Urban Suburban Parochial Private Total 



Who finds it Most gsefull 
Ce ntr a l Office 

Superintendent - - 1(2Z) 

General iMmijiistratdr - 4(16%) J<38%) 7(14%) 
Instructional 

_ Administrator 3(19Z5 4(16%) 1(50Z5 - 8(i6Z) 
Pupil Personnel 

Administrator - 1(4Z) - - 1(2%) 
Buildings 

Principals ^ . 2(13%) 6(24%) - 8ri6%l 

Teachers 9(56%) 7(28%) 1(50%) 3(38%) 20(41%) 

Counselors 1(6%) 2(8%) - 2(25%) 5(40%) 

Other . 1(6%) 1(4%) - - 2(4%) 



resspdnses n * 16 25 2 8 51 



Who Would mss 

Testing the Most 
^ f It Were A boMshed? 

Central Office 

Board 1(4%) 3(7%) - - 4(5%) 

Superintendent - 1(3%) - - 1(1%) 

. General ; 

Administrator 6(21%) 3(7%) - 1(ZD%) 10(13%) 

Instructional 

_Administratdr_ 2(7%) 3(7%) 1(50%) - 6(8%) 

Pupil Personnel -.^ 

Administrator - - - - 0(O%X 

' Building 

principals 1(4%) - ^ ^ - 1(1%) _ 

Teachers ^ 8(29%) 11(29%) - 1(20%) 20(27%) 

Counselors 2(7%) 8(20%) 1(50%) - 11(15%) 
Special Education 

and Itinerant Staff 1(4%) 3(7%) - - 4(5%) 
Parents^ students^ _ . ^ 

& community 4(14%) 6(15%) - 3(60%) 13(17%) 

Other I 3(11%) 2(5%) - - 5(7%) 



responses n - ' 28 40 2 5 75 



Note: Percentages may iidt sum to 100 due to rounding. 



Table 7 



Behaviors 



Inforina tton Processing Behaviors Employed by 
Central Of f ice ^Adai^lstratbrs in Viewing Test Scores 

School System Type 



Urban Suburban Parochial Private 



Interpretation Rules 

Intra time periba 
Internal Reference 

Group 
External Reference 

Group 
Non-comparative 

Jnter time Period 
Internal Reference 
Group 

Other (Gombination 
of above) 

responses n " 

What kinds of Scores 
Would Cause Central 
Office Action? 



Deviation from expectations 
derived from: 



1(10%) 5(9Z) 

6(60%) 6(11%1 
35(65%) 



3(30%) 8(15%) 
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Intra time period 
Internal Reference 

Group 
External Reference 
r Group 

NoQ-compar a t ive 

— inter time period 
Internal Reference 
Group 

Other (Combination 
of above) 



responses n 



- ' 6(32%) 

8(100%) 6(32%) 
1(5%) 



2(11%) 
4(21%) 



1(100%) 



1(50%) 
1(50%) 



3(19%) 



li 

7(44%) 



1(6%) 

4(25%) 

16 



4(44%) 

1(11%) 
3(33%) 



l(llZ) 



Total 



10(12%) 

13(16%) 
42(52%) 



12(15%) 

4(5%) 

81 



16(26%) 

16(42%) 
5(13%) 



8 



19 



5(13%) 



38 
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